The Sample Complexity of Teaching by Reinforcement on Q-Learning

We study the sample complexity of teaching, termed as "teaching dimension" (TDim) in literature, for teaching-by-reinforcement paradigm, where teacher guides student through rewards. This is distinct from teaching-by-demonstration paradigm motivated by robotics applications, teaches providing demonstrations state/action trajectories. The applies to a wider range real-world settings demonstration inconvenient, but has not been studied systematically. In this paper, we focus on specific family reinforcement learning algorithms, Q-learning, and characterize TDim under different teachers with varying control power over environment, present matching optimal teaching algorithms. Our results provide minimum number samples needed learning, discuss their connections standard PAC-style RL results. algorithms have potential speed up agent applications helpful available.